End-to-end principle
}} The end-to-end principle is a design framework in computer networking. In networks designed according to this principle, application-specific features reside in the communicating end nodes of the network, rather than in intermediary nodes, such as gateways and routers, that exist to establish the network. The end-to-end principle originated in the work by Paul Baran in the 1960s, which addressed the requirement of network reliability when the building blocks are inherently unreliable. It was first articulated explicitly in 1981 by Saltzer, Reed, and Clark. The meaning of the end-to-end principle has been continuously reinterpreted ever since its initial articulation. Also, noteworthy formulations of the end-to-end principle can be found prior to the seminal 1981 Saltzer, Reed, and Clark paper. A basic premise of the principle is that the payoffs from adding features to a simple network quickly diminish, especially in cases in which the end hosts have to implement those functions only for reasons of conformance, i.e. completeness and correctness based on a specification. Implementing a specific function incurs some resource penalties regardless of whether the function is used or not, and implementing a specific function in the network distributes these penalties among all clients. The end-to-end principle is closely related, and sometimes seen as a direct precursor to the principle of net neutrality. Category:Kvng RTH Concept The fundamental notion behind the end-to-end principle is that for two processes communicating with each other via some communication means, the reliability obtained from that means cannot be expected to be perfectly aligned with the reliability requirements of the processes. In particular, meeting or exceeding very high reliability requirements of communicating processes separated by networks of nontrivial size is more costly than obtaining the required degree of reliability by positive end-to-end acknowledgements and retransmissions (referred to as PAR or ARQ). "attention to reliability at higher levels is required regardless of the control strategy of the network".|group=nb}} Put differently, it is far easier and more tractable to obtain reliability beyond a certain margin by mechanisms in the end hosts of a network rather than in the intermediary nodes, especially when the latter are beyond the control of and accountability to the former.The possibility of enforceable contractual remedies notwithstanding, it is impossible for any network in which intermediary resources are shared in a non-deterministic fashion to guarantee perfect reliability. At most, it may quote statistical performance averages. An end-to-end PAR protocol with infinite retries can obtain arbitrarily high reliability from any network with a higher than zero probability of successfully transmitting data from one end to another. The end-to-end principle does not trivially extend to functions beyond end-to-end error control and correction. E.g., no straightforward end-to-end arguments can be made for communication parameters such as latency and throughput. Based on a personal communication with Saltzer (lead author of the original end-to-end paper) Blumenthal and Clark in a 2001 paper note: From the beginning, the end-to-end arguments revolved around requirements that could be implemented correctly at the end-points; if implementation inside the network is the only way to accomplish the requirement, then an end-to-end argument isn't appropriate in the first place. The basic notion: reliability from unreliable parts In the 1960s, Paul Baran and Donald Davies in their pre-ARPANET elaborations of networking made brief comments about reliability that capture the essence of the later end-to-end principle. To quote from a 1964 Baran paper: Reliability and raw error rates are secondary. The network must be built with the expectation of heavy damage anyway. Powerful error removal methods exist. (p. 5). Similarly, Davies notes on end-to-end error control: It is thought that all users of the network will provide themselves with some kind of error control and that without difficulty this could be made to show up a missing packet. Because of this, loss of packets, if it is sufficiently rare, can be tolerated. (p. 2.3). Early trade-offs: experiences in the ARPANET The ARPANET was the first large-scale general-purpose packet switching network implementing several of the basic notions previously touched on by Baran and Davies, and demonstrating several important aspects of the end-to-end principle: ;Packet switching pushes some logical functions toward the communication end points: If the basic premise of a distributed network is packet switching, then functions such as reordering and duplicate detection inevitably have to be implemented at the logical end points of such network. Consequently, the ARPANET featured two distinct levels of functionality – (1) a lower level concerned with transporting data packets between neighboring network nodes (called IMPs), and (2) a higher level concerned with various end-to-end aspects of the data transmission. Dave Clark, one of the authors of the end-to-end principle paper, concludes: "The discovery of packets is not a consequence of the end-to-end argument. It is the success of packets that make the end-to-end argument relevant" (slide 31). ;No arbitrarily reliable data transfer without end-to-end acknowledgment and re-transmission mechanisms: The ARPANET was designed to provide reliable data transport between any two end points of the network much like a simple I/O channel between a computer and a nearby peripheral device. In order to remedy any potential failures of packet transmission normal ARPANET messages were handed from one node to the next node with a positive acknowledgment and retransmission scheme; after a successful handover they were then discarded, no source-to-destination re-transmission in case of packet loss was catered for. However, in spite of significant efforts, perfect reliability as envisaged in the initial ARPANET specification turned out to be impossible to provide a reality that became increasingly obvious once the ARPANET grew well beyond its initial four node topology. The ARPANET thus provided a strong case for the inherent limits of network based hop-by-hop reliability mechanisms in pursuit of true end-to-end reliability. those would have added considerable complexity to the common host level protocol (Host-Host Protocol). While the desirability of host-host reliability mechanisms was articulated in , after some discussion they were dispensed with (although higher level protocols or applications were, of course, free to implement such mechanisms themselves). For a recount of the debate at the time see Bärwolff 2010, pp. 56-58 and the notes therein, especially notes 151 and 163.|group=nb}} ;Trade-off between reliability, latency, and throughput: The pursuit of perfect reliability may hurt other relevant parameters of a data transmission most importantly latency and throughput. This is particularly important for applications that require no perfect reliability, but rather value predictable throughput and low latency the classic example being interactive real-time voice applications. This use case was catered for in the ARPANET by providing a raw message service that dispensed with various reliability measures so as to provide faster and lower latency data transmission service to the end hosts. (p. 2), in 1974 BBN introduced the raw message service (Raw Message Interface, RMI) to the ARPANET, primarily in order to allow hosts to experiment with packet voice applications, but also acknowledging the use of such facility in view of possibly internetwork communication (cf. a BBN Report 2913 at pp. 55 f.). See also Bärwolff 2010, pp. 80-84 and the copious notes therein.|group=nb}} The canonical case: TCP/IP On the Internet the Internet Protocol a connectionless datagram service with no delivery guarantees and effectively no QoS parameters is used for nearly all communications. Arbitrary protocols may sit on top of IP. It turns out that some applications (such as voice, in many cases) do not need reliable retransmission, and so the only reliability in IP is in the checksum of the IP header (which is necessary to prevent bit errors from sending packets on wild routing paths.) End-to-end acknowledgment and retransmission is relegated to the connection-oriented TCP which sits on top of IP. The functional split between IP and TCP exemplifies proper application of the end-to-end principle to transport protocol design. Example An example of the end-to-end principle is that of an arbitrarily reliable file transfer between two end-points in a distributed network of a varying, nontrivial size: The only way two end-points can obtain a completely reliable transfer is by transmitting and acknowledging a checksum for the entire data stream; in such a setting, lesser checksum and acknowledgement (ACK/NACK) protocols are justified only for the purpose of optimizing performance - they are useful to the vast majority of clients, but are not enough to fulfil the reliability requirement of this particular application. Thorough checksum is hence best done at the end-points, and the network maintains a relatively low level of complexity and reasonable performance for all clients. Limitations The most important limitation of the end-to-end principle is that its basic conclusion, placing functions in the application end points rather than in the intermediary nodes, is not trivial to operationalize. Specifically: * The principle assumes a notion of distinct application end points as opposed to intermediary nodes, which makes little sense when considering the structure of distributed applications; * The principle assumes a dichotomy between non-application-specific and application-specific functions, the former to be part of the operations between application end points and the latter to be implemented by the application end points themselves, while arguably no function to be performed in a network is fully orthogonal to all possible application needs; * The principle is silent on functions that may not be implemented completely and correctly in the application end points and places no upper bound on the amount of application specific functions that may be placed with intermediary nodes on grounds of performance considerations, or economic trade-offs. An example of the limitations of the end-to-end principle exists in mobile devices. Pushing service-specific complexity to the endpoints can cause issues with mobile devices if the device has unreliable access to network channels. Further problems can be seen with a decrease in network transparency from the addition of network address translation. The implementation of new protocols such as Internet Protocol version 6 also has potential consequences for the end-to-end principle. IPv4 relied on network address translation to combat address exhaustion. With the introduction of IPv6, users once again have unique identifiers, allowing for true end-to-end connectivity. Unique identifiers are typically based on physical addresses, or can be generated by the host. Notes References . Historic. NWG. Scheblik, T. J., D. B. Dawkins, and Advanced Research Projects Agency (1968). RFQ for ARPA Computer Network. Request for Quotations. Advanced Research Projects Agency (ARPA), Department of Defense (DoD). (Online copy). Saltzer, J. H., D. P. Reed, and D. D. Clark (1981) "End-to-End Arguments in System Design". In: Proceedings of the Second International Conference on Distributed Computing Systems. Paris, France. April 8–10, 1981. IEEE Computer Society, pp. 509-512. Saltzer, J. H., D. P. Reed, and D. D. Clark (1984) "End-to-End Arguments in System Design". In: ACM Transactions on Computer Systems 2.4, pp. 277-288. (See also here for a version from Saltzer's MIT homepage.) Sunshine, C. A. (1975). Issues in Communication Protocol Design – Formal Correctness. Draft. INWG Protocol Note 5. IFIP WG 6.1 (INWG). (Copy from CBI). Walden, D. C. (1972). "The Interface Message Processor, Its Algorithms, and Their Implementation". In: AFCET Journées d’Études: Réseaux de Calculateurs (AFCET Workshop on Computer Networks). Paris, France. May 25–26, 1972. Association Française pour la Cybernétique Économique et Technique (AFCET). (Online copy). Metcalfe, R. M. (1973). "Packet Communication". PhD thesis. Cambridge, MA: Harvard University. Online copy (revised edition, published as MIT Laboratory for Computer Science Technical Report 114). Mostly written at MIT Project MAC and Xerox PARC. BBN (1974). Interface Message Processors for the Arpa Computer Network. BBN Report 2913. Quarterly Technical Report No. 7, 1 July 1974 to 30 September 1974. Bolt, Beranek and Newman Inc. (BBN). Walden, D. C. (1974) Some Changes to the IMP and the IMP/Host Interface. . Historic. NWG. Saltzer, J. H. (1980). End-to-End Arguments in System Design. Request for Comments No. 185, MIT Laboratory for Computer Science, Computer Systems Research Division. (Online copy). }} External links *MIT homepage of Jerome H. Saltzer featuring publication list, working papers, biography, etc. *Personal homepage of David P. Reed featuring publication list, blog, biography, etc. *MIT homepage of David D. Clark featuring publication list, working papers, biography, etc. Category:Internet architecture Category:Network architecture Category:Net neutrality Category:Programming paradigms